Skip to content

Add npm publishing and update documentation#719

Draft
jbrejner wants to merge 9 commits intomainfrom
4590_cli-as-npm-package
Draft

Add npm publishing and update documentation#719
jbrejner wants to merge 9 commits intomainfrom
4590_cli-as-npm-package

Conversation

@jbrejner
Copy link
Copy Markdown
Contributor

@jbrejner jbrejner commented Mar 23, 2026

Distribute kosli cli as npm package.

Install globally with npm install -g @kosli/cli
The above will install the wrapper package which detects the current platform and architecture and then pull the appriate package with the correct binary.

Draft because I still need TODO

  • add api token for publishing to npm registry
  • kosli attest on the packages created
  • have @jbpros to take a look at packages. I currently have snapshot packages at @jbrejner/cli - are there things a good npm package should have ?

jbrejner and others added 5 commits March 21, 2026 19:22
If gorelease in not in "Release" mode, npm packages are not pushed, so we can avoid multitudes of snapshot packages being pushed to public registry
The hook runs after. If goreaser is not in release mode, npm will build in dry-run mode, so the packages are not pushed to registry
@jbrejner jbrejner requested a review from pbeckham March 23, 2026 17:30
@AlexKantor87
Copy link
Copy Markdown
Contributor

PR #719 Multi-Agent Review: "Add npm publishing and update documentation"

Repository: kosli-dev/cli | Author: jbrejner | Status: DRAFT Branch: 4590_cli-as-npm-packagemain Stats: +503 / -5 across 30 files | 5 commits PR URL: https://github.com//pull/719


The Review Panel

Four agents with distinct personas reviewed this PR independently, then their findings were synthesized. Here's who was on the panel:

Agent Persona Focus Area
Sasha Security Engineer — paranoid, cynical, thinks in attack surfaces Secrets management, supply chain risks, token handling
Dev DevOps/CI Engineer — battle-hardened, pragmatic, "what fails at 2am?" GoReleaser integration, pipeline reliability, failure modes
Priya NPM Packaging Expert — meticulous, encyclopedic npm knowledge Package.json correctness, DX, npm ecosystem patterns
Doc Technical Writer & DX Advocate — empathetic, user-journey focused Documentation completeness, onboarding experience, error UX

Unanimous Verdict: DO NOT MERGE (yet)

All four agents independently reached the same conclusion: the PR has a sound architectural foundation (the esbuild-style per-platform binary distribution pattern is well-chosen) but has several blocking issues that would result in a non-functional npm package if published today. This aligns with the PR's own DRAFT status and the author's listed TODOs.


The Big Three: Issues Every Agent Flagged

These three issues were independently identified by all four reviewers, making them the highest-confidence findings:

1. The Missing bin/kosli JS Shim (flagged by: ALL FOUR)

The wrapper's package.json declares "bin": {"kosli": "bin/kosli"}, but the JavaScript shim file that should live at npm/wrapper/bin/kosli is nowhere in the diff. Worse, .gitignore adds npm/wrapper/bin/*, which would prevent committing it.

  • Sasha (Security): "Users get command not found — package is non-functional"
  • Dev (CI/CD): "No evidence the shim exists or is tested"
  • Priya (NPM): "This is the most critical issue — every installation will be broken"
  • Doc (DX): "The README describes a file that doesn't exist in the repo. Is it generated? By what?"

Consensus: This is likely an oversight where the shim was developed locally but blocked by the gitignore pattern. The shim must be a committed JS file (not a binary), so it shouldn't be gitignored.

2. Token Variable Mismatch (flagged by: Sasha, Dev, Priya)

The .npmrc files reference ${MY_LOCAL_NPM_TOKEN}, but the CI workflow sets NPM_TOKEN. The publish script doesn't bridge this gap.

  • Sasha: "npm publishing will FAIL due to authentication errors — incomplete implementation"
  • Dev: "Silent npm ERR! 401 Unauthorized failures in CI"
  • Priya: "Confusion about where tokens go; risk of accidentally committing tokens"

Consensus: The token naming needs to be unified, or the publish script needs to explicitly set the npm auth token from the CI environment variable.

3. Silent Postinstall Failures (flagged by: ALL FOUR)

The install.js postinstall script exits with code 0 on every failure path — unsupported platform, missing binary, failed validation. Users get no clear signal their installation is broken.

  • Sasha: "Weak validation — a trojanized binary that still executes passes silently"
  • Dev: "No binary validation — corrupt/truncated binaries could ship"
  • Priya: "Violates the postinstall contract. Users will debug for hours"
  • Doc: "No troubleshooting section for the errors this script silently swallows"

Consensus: At minimum, the script should exit non-zero when the binary is missing or fails validation on a supported platform. The current "never fail" approach is too permissive.


Where the Agents Diverged: The Interesting Debates

The sed -i Question

Dev and Priya both flagged the use of sed -i in npm-publish.sh to mutate package.json files, but for different reasons:

  • Dev worried about atomicity: "If sed fails partway, repo is left corrupted. No backup."
  • Priya worried about portability: "macOS BSD sed needs sed -i '', Linux doesn't. This will break local dev."

Sasha didn't flag this — from a security perspective, sed in a CI pipeline is unremarkable. Doc didn't flag it either — it's invisible to users.

The nfpms Template Change

Dev was the only agent to catch that the .goreleaser.yml nfpms section removed the {{ else if eq .Arch "arm64" }}{{ .Arch }}_v8.0 clause. Dev called this "undocumented arm64_v8.0 suffix removal" and questioned whether it was intentional or a regression affecting Linux ARM64 package builds.

The other three agents didn't notice this change buried in the goreleaser config — it's a subtle, domain-specific issue that only a CI specialist would spot.

Supply Chain Irony

Sasha made an observation unique to the security review: "No SHA256 verification, no SLSA attestation verification — ironic given Kosli IS a compliance tool." This philosophical point — that a compliance tool's own distribution lacks the verification it recommends — wasn't raised by the others.

Error Messages vs. Error Handling

Doc and Priya had subtly different takes on the postinstall problem:

  • Priya wanted the script to exit non-zero (break the install if things are wrong)
  • Doc wanted better error messages and a troubleshooting guide (help users fix it)

These aren't contradictory — you need both — but they reflect the different priorities of a package maintainer vs. a documentation writer.


Per-Agent Unique Findings

Sasha (Security) — Unique Catches

  • Attack surface analysis: Identified 5 new supply chain vectors (platform substitution, wrapper hijacking, typosquatting, token leakage, CI escape)
  • .npmrc files as a pattern risk: Even with variable placeholders, committing .npmrc files establishes a dangerous pattern for future contributors

Dev (DevOps) — Unique Catches

  • No rollback on partial publish: If npm publish fails on package 3 of 7, the script continues. Result: broken release with some platforms available, others missing
  • Cleanup hooks don't verify success: The before hooks assume deletion succeeded but don't check for permission issues
  • GoReleaser after hook conditional: Noted the snapshot/dry-run conditional is well-designed

Priya (NPM) — Unique Catches

  • Missing engines field: No Node.js version constraint in any package.json
  • Missing repository.directory: Monorepo packages should include "directory": "npm/wrapper" for npm registry navigation
  • Missing trailing newlines: Several package.json files lack POSIX-standard trailing newlines (will cause linter warnings and potential npm auto-fix diffs)
  • 0.0.0 version semantics: npm interprets "0.0.0" as an exact pin, not "any version" — the publish script must update these atomically

Doc (Technical Writer) — Unique Catches

  • Frontmatter syntax error: Line 1 of install.md changed --- to --- (leading space) — will break Hugo rendering
  • No npx documentation: Users discovering the package on npmjs.org will naturally try npx @kosli/cli — is this supported?
  • Tree diagram formatting errors: The README has two └── at the same indentation level (cosmetic but sloppy)
  • Per-platform READMEs need "install @kosli/cli instead" guidance: Users who find @kosli/cli-linux-x64 on npm need to be redirected

Consolidated Recommendations (Priority Order)

Must-fix before merge

  1. Commit the bin/kosli JS wrapper shim (and fix .gitignore to allow it)
  2. Align token variable naming between .npmrc and CI (NPM_TOKEN vs MY_LOCAL_NPM_TOKEN)
  3. Fix install.md frontmatter leading space (line 1)
  4. Make postinstall exit non-zero when binary is missing/broken on supported platforms
  5. Add error handling for partial publish failures in npm-publish.sh
  6. Confirm the nfpms arm64 template change is intentional

Should-fix

  1. Add "engines": { "node": ">=16.0.0" } to wrapper package.json
  2. Add trailing newlines to all package.json files
  3. Add troubleshooting section to install docs
  4. Document Node.js version requirements in the NPM install tab
  5. Fix tree diagram formatting in npm/README.md
  6. Add repository.directory to wrapper package.json

Nice-to-have

  1. Add binary integrity verification (checksums)
  2. Document npx behavior
  3. Improve per-platform README content
  4. Add CI test step that actually installs and runs the npm package

Process Observations

What worked well about multi-agent review

  • Coverage breadth: Each agent caught issues the others missed (Dev's nfpms regression, Doc's frontmatter error, Priya's engines field, Sasha's supply chain vectors)
  • Confidence calibration: Issues flagged by 3-4 agents are clearly high-priority; issues flagged by only 1 agent are worth investigating but may be lower priority
  • Perspective diversity: The same issue (silent postinstall) was analyzed from four angles — security (attack surface), reliability (failure modes), correctness (npm contract), and UX (user confusion)

What was interesting about the disagreements

  • Agents naturally weighted the same evidence differently based on their expertise
  • No actual contradictions — disagreements were about priority, not facts
  • The "overlap zone" between agents was the most valuable (3-4 agents flagging the same issue independently = high confidence)

Review conducted: 2026-03-24 PR URL: https://github.com//pull/719

@claude
Copy link
Copy Markdown
Contributor

claude bot commented Mar 25, 2026

Claude finished @jbrejner's task in 2m 54s —— View job


PR Review: Add npm publishing and update documentation

  • Run static analysis (go build, go vet, go mod tidy)
  • Review changed files
  • Post inline comments
  • Post summary

Static Analysis

All Go checks pass: go build ./..., go vet ./..., and go mod tidy report no issues. No Go code was changed in this PR.

Issues Addressed Since Multi-Agent Review

Good progress — the three major issues flagged by all reviewers have been fixed:

  • bin/kosli JS shim — now committed (ee516b12)
  • Token variable mismatch.npmrc files now use ${NPM_TOKEN} (2291d409)
  • Silent postinstall failuresinstall.js now exits non-zero when binary is missing or validation fails (899f001e)

Remaining Issues

Must-fix:

  1. install.md frontmatter broken — Line 1 has a leading space ( --- instead of ---), which will break Hugo rendering. (inline comment)

  2. npm-publish.sh subshell swallows failures — The find | while read pipeline runs the loop in a subshell, so set -e won't catch npm pack/npm publish failures inside it. A partial publish could go unnoticed. (inline comment)

Should-confirm:

  1. nfpms arm64 template change — The _v8.0 suffix was removed from the contents src path. If GoReleaser still generates the _v8.0 directory, Linux ARM64 .deb/.rpm packages will break. (inline comment)

Nice-to-have:

  1. Unsupported platform exits 0 in install.js — consider exiting non-zero to avoid silent CI failures. (inline comment)

  2. Missing engines field in npm/wrapper/package.json — adding "engines": { "node": ">=16.0.0" } gives users on old Node versions a clear error.

  3. Tree diagram in npm/README.md — the last two entries both use └── at the same indent level; the first should be ├──.

What Looks Good

  • The esbuild-style per-platform binary distribution is the right pattern — well-proven in the ecosystem.
  • The GoReleaser post-build hooks that copy binaries into npm package directories are clean and well-structured.
  • The dry-run/snapshot conditional in the after hook ({{ if or .IsSnapshot (not .IsRelease) }} --dry-run{{ end }}) is a nice touch.
  • The bin/kosli shim (npm/wrapper/bin/kosli) is clean — proper error handling, exit code passthrough, and concise platform detection.
  • The npm/README.md is thorough internal documentation that will help future maintainers understand the packaging setup.

Comment on lines +42 to +51
find npm -name package.json ! -path "npm/wrapper/*" | while read -r f; do
PKG_DIR="$(dirname "$f")"
PKG_NAME="$(basename "$PKG_DIR")"
echo "Packing ${PKG_NAME}..."
(cd "$PKG_DIR" && npm pack)
if [ "$DRY_RUN" = false ]; then
echo "Publishing ${PKG_NAME}..."
(cd "$PKG_DIR" && npm publish --tag "$NPM_TAG")
fi
done
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Partial publish leaves broken release. If one platform package fails to publish, set -e will abort the loop — but packages already published remain on npm with no matching wrapper. Users on those platforms will get the binary but the wrapper (with the bin shim) won't exist at that version.

Consider either:

  1. Publishing all packages in a first pass (npm pack only), then a second pass for npm publish — so you catch pack failures early.
  2. Or at minimum, adding cleanup/retry logic so a partial publish doesn't leave the registry in an inconsistent state.

Also: the find | while read pipeline means set -e won't catch failures inside the loop on some bash versions (the loop runs in a subshell). Consider using a for loop with a glob or xargs instead.

fi

# Inject version into all platform package.json files
find npm -name package.json -exec sed -i "s/\"version\": \"[^\"]*\"/\"version\": \"${VERSION}\"/" {} \;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Portability issue with sed -i. On macOS, BSD sed requires sed -i '' (empty backup extension), while GNU sed on Linux uses sed -i without an argument. This will break local development on macOS.

Consider using a portable pattern:

Suggested change
find npm -name package.json -exec sed -i "s/\"version\": \"[^\"]*\"/\"version\": \"${VERSION}\"/" {} \;
find npm -name package.json -exec sed -i.bak "s/\"version\": \"[^\"]*\"/\"version\": \"${VERSION}\"/" {} \; -exec rm {}.bak \;

Or better yet, use node -e or jq to manipulate JSON properly instead of regex on JSON files.

contents:
# The src and dst attributes also supports name templates
- src: dist/{{ .ProjectName }}_{{ .Os }}_{{ if .Amd64 }}{{ .Arch }}_v1{{ else if .Arm }}{{ .Arch }}_6{{ else if eq .Arch "arm64" }}{{ .Arch }}_v8.0{{ else }}{{ .Arch }}{{ end }}/kosli
- src: dist/{{ .ProjectName }}_{{ .Os }}_{{ if .Amd64 }}{{ .Arch }}_v1{{ else if .Arm }}{{ .Arch }}_6{{ else }}{{ .Arch }}{{ end }}/kosli
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intentional change? This removes the {{ else if eq .Arch "arm64" }}{{ .Arch }}_v8.0 clause from the nfpms contents.src template. This changes the expected path for arm64 Linux .deb/.rpm packages from dist/kosli_linux_arm64_v8.0/kosli to dist/kosli_linux_arm64/kosli.

If GoReleaser still produces the _v8.0 suffixed directory, this will break Linux ARM64 .deb/.rpm packaging. Please confirm this matches the directory names GoReleaser actually creates with the version you're using.

Comment on lines +19 to +24
if (!SUPPORTED[platform] || !SUPPORTED[platform][arch]) {
process.stderr.write(
`[kosli] Note: ${platform}/${arch} is not a supported platform.\n` +
`[kosli] The kosli binary will not be available on this system.\n`
);
process.exit(0);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: should unsupported platforms really exit 0? This means npm install @kosli/cli will "succeed" on unsupported platforms with no indication of failure in the exit code. A CI pipeline installing kosli on an unsupported platform would silently succeed, then fail later when trying to run kosli.

The bin/kosli shim already exits non-zero for unsupported platforms, so the postinstall could too — users who intentionally want to skip would use --ignore-scripts.

"registry": "https://registry.npmjs.org",
"access": "public"
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing engines field. Consider adding a minimum Node.js version constraint:

"engines": { "node": ">=16.0.0" }

This helps users on old Node versions get a clear error rather than cryptic failures. Node 16+ is reasonable since require.resolve and spawnSync are well-established by then.

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
@@ -1,4 +1,4 @@
---
---
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Leading space in frontmatter delimiter. Line 1 is --- (with a leading space) instead of ---. This will break Hugo's frontmatter parsing — the page won't render correctly.

Suggested change
---
---

Comment on lines +42 to +51
find npm -name package.json ! -path "npm/wrapper/*" | while read -r f; do
PKG_DIR="$(dirname "$f")"
PKG_NAME="$(basename "$PKG_DIR")"
echo "Packing ${PKG_NAME}..."
(cd "$PKG_DIR" && npm pack)
if [ "$DRY_RUN" = false ]; then
echo "Publishing ${PKG_NAME}..."
(cd "$PKG_DIR" && npm publish --tag "$NPM_TAG")
fi
done
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: set -e doesn't catch failures inside find | while read pipeline. The while loop runs in a subshell (due to the pipe), so if npm pack or npm publish fails, the script will continue and the exit code is lost. This means a partial publish can happen silently.

Replace with a for loop over a glob or use process substitution:

Suggested change
find npm -name package.json ! -path "npm/wrapper/*" | while read -r f; do
PKG_DIR="$(dirname "$f")"
PKG_NAME="$(basename "$PKG_DIR")"
echo "Packing ${PKG_NAME}..."
(cd "$PKG_DIR" && npm pack)
if [ "$DRY_RUN" = false ]; then
echo "Publishing ${PKG_NAME}..."
(cd "$PKG_DIR" && npm publish --tag "$NPM_TAG")
fi
done
while IFS= read -r f; do
PKG_DIR="$(dirname "$f")"
PKG_NAME="$(basename "$PKG_DIR")"
echo "Packing ${PKG_NAME}..."
(cd "$PKG_DIR" && npm pack)
if [ "$DRY_RUN" = false ]; then
echo "Publishing ${PKG_NAME}..."
(cd "$PKG_DIR" && npm publish --tag "$NPM_TAG")
fi
done < <(find npm -name package.json ! -path "npm/wrapper/*")

Or simpler: use a for loop with globbing: for f in npm/cli-*/package.json; do ...

contents:
# The src and dst attributes also supports name templates
- src: dist/{{ .ProjectName }}_{{ .Os }}_{{ if .Amd64 }}{{ .Arch }}_v1{{ else if .Arm }}{{ .Arch }}_6{{ else if eq .Arch "arm64" }}{{ .Arch }}_v8.0{{ else }}{{ .Arch }}{{ end }}/kosli
- src: dist/{{ .ProjectName }}_{{ .Os }}_{{ if .Amd64 }}{{ .Arch }}_v1{{ else if .Arm }}{{ .Arch }}_6{{ else }}{{ .Arch }}{{ end }}/kosli
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Is the arm64 _v8.0 suffix removal intentional? The previous template had {{ else if eq .Arch "arm64" }}{{ .Arch }}_v8.0 which matched GoReleaser's default directory naming for arm64 builds. Removing it changes the expected path from dist/kosli_linux_arm64_v8.0/kosli to dist/kosli_linux_arm64/kosli.

This was also flagged in commit ee7fb466 ("Update goreleaser config to fix arm64 package #711"). If your GoReleaser version no longer produces the _v8.0 suffix, this is fine — but please confirm this matches the actual directory names GoReleaser creates, otherwise Linux ARM64 .deb/.rpm packaging will break.

Comment on lines +19 to +24
if (!SUPPORTED[platform] || !SUPPORTED[platform][arch]) {
process.stderr.write(
`[kosli] Note: ${platform}/${arch} is not a supported platform.\n` +
`[kosli] The kosli binary will not be available on this system.\n`
);
process.exit(0);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider exiting non-zero for unsupported platforms. Currently this exits 0, which means npm install @kosli/cli will "succeed" on unsupported platforms. A CI pipeline would silently succeed here and only fail later when trying to run kosli. The bin/kosli shim already exits non-zero for unsupported platforms — the postinstall could match that behavior. Users who intentionally want to skip can use --ignore-scripts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants